Sound Event Detection by Exploring Audio Sequence Modelling

Everyday sounds in real-world environments are a powerful source of information by which humans can interact with their environments. Humans can infer what is happening around them by listening to everyday sounds. At the same time, it is a challenging task for a computer algorithm in a smart device to automatically recognise, understand, and interpret everyday sounds. Sound event detection (SED) is the process of transcribing an audio recording into sound event tags with onset and offset time values. This involves classification and segmentation of sound events in the given audio recording. SED has numerous applications in everyday life which include security and surveillance, automation, healthcare monitoring, multimedia information retrieval, and assisted living technologies. SED is to everyday sounds what automatic speech recognition (ASR) is to speech and automatic music transcription (AMT) is to music. The fundamental questions in designing a sound recognition system are, which portion of a sound event should the system analyse, and what proportion of a sound event should the system process in order to claim a confident detection of that particular sound event. While the classification of sound events has improved a lot in recent years, it is considered that the temporal-segmentation of sound events has not improved in the same extent. The aim of this thesis is to propose and develop methods to improve the segmentation and classification of everyday sound events in SED models. In particular, this thesis explores the segmentation of sound events by investigating audio sequence encoding-based and audio sequence modelling-based methods, in an effort to improve the overall sound event detection performance. In the first phase of this thesis, efforts are put towards improving sound event detection by explicitly conditioning the audio sequence representations of an SED model using sound activity detection (SAD) and onset detection. To achieve this, we propose multi-task learning-based SED models in which SAD and onset detection are used as auxiliary tasks for the SED task. The next part of this thesis explores self-attention-based audio sequence modelling, which aggregates audio representations based on temporal relations within and between sound events, scored on the basis of the similarity of sound event portions in audio event sequences. We propose SED models that include memory-controlled, adaptive, dynamic, and source separation-induced self-attention variants, with the aim to improve overall sound recognition

A Multi-Objective Optimal Experimental Design Framework for Enhancing the Efficiency of Online Model-Identification Platforms

Recent advances in automation and digitization enable the close integration of physical devices with their virtual counterparts, facilitating the real-time modeling and optimization of a multitude of processes in an automatic way. The rich and continuously updated data environment provided by such systems makes it possible for decisions to be made over time to drive the process toward optimal targets. In many manufacturing processes, in order to achieve an overall optimal process, the simultaneous assessment of multiple objective functions related to process performance and cost is necessary. In this work, a multi-objective optimal experimental design framework is proposed to enhance the efficiency of online model-identification platforms. The proposed framework permits flexibility in the choice of trade-off experimental design solutions, which are calculated online—that is, during the execution of experiments. The application of this framework to improve the online identification of kinetic models in flow reactors is illustrated using a case study in which a kinetic model is identified for the esterification of benzoic acid (BA) and ethanol in a microreactor

Closed-Loop Model-Based Design of Experiments for Kinetic Model Discrimination and Parameter Estimation: Benzoic Acid Esterification on a Heterogeneous Catalyst

An autonomous reactor platform was developed to rapidly identify a kinetic model for the esterification of benzoic acid with ethanol with the heterogeneous Amberlyst-15 catalyst. A five-step methodology for kinetic studies was employed to systematically reduce the number of experiments required to identify a practical kinetic model. This included (i) initial screening using traditional factorial designed steady-state experiments, (ii) proposing and testing candidate kinetic models, (iii) performing an identifiability analysis to reject models whose model parameters cannot be estimated for a given experimental budget, (iv) performing online Model-Based Design of Experiments (MBDoE) for model discrimination to identify the best model from a list of candidates, and (v) performing online MBDoE for improving parameter precision for the chosen model. This methodology combined with the reactor platform, which conducted all kinetic experiments unattended, reduces the number of experiments and time required to identify kinetic models, significantly increasing lab productivity

Clindamycin-modified Triple Antibiotic Nanofibers: A Stain-free Antimicrobial Intracanal Drug Delivery System

INTRODUCTION: A biocompatible strategy to promote bacterial eradication within the root canal system after pulpal necrosis of immature permanent teeth is critical to the success of regenerative endodontic procedures. This study sought to synthesize clindamycin-modified triple antibiotic (metronidazole, ciprofloxacin, and clindamycin [CLIN]) polymer (polydioxanone [PDS]) nanofibers and determine in vitro their antimicrobial properties, cell compatibility, and dentin discoloration. METHODS: CLIN-only and triple antibiotic CLIN-modified (CLIN-m, minocycline-free) nanofibers were processed via electrospinning. Scanning electron microscopy, Fourier-transform infrared spectroscopy (FTIR), and tensile testing were performed to investigate fiber morphology, antibiotic incorporation, and mechanical strength, respectively. Antimicrobial properties of CLIN-only and CLIN-m nanofibers were assessed against several bacterial species by direct nanofiber/bacteria contact and over time based on aliquot collection up to 21 days. Cytocompatibility was measured against human dental pulp stem cells. Dentin discoloration upon nanofiber exposure was qualitatively recorded over time. The data were statistically analyzed (P < .05). RESULTS: The mean fiber diameter of CLIN-containing nanofibers ranged between 352 ± 128 nm and 349 ± 128 nm and was significantly smaller than PDS fibers. FTIR analysis confirmed the presence of antibiotics in the nanofibers. Hydrated CLIN-m nanofibers showed similar tensile strength to antibiotic-free (PDS) nanofibers. All CLIN-containing nanofibers and aliquots demonstrated pronounced antimicrobial activity against all bacteria. Antibiotic-containing aliquots led to a slight reduction in dental pulp stem cell viability but were not considered toxic. No visible dentin discoloration upon CLIN-containing nanofiber exposure was observed. CONCLUSIONS: Collectively, based on the remarkable antimicrobial effects, cell-friendly, and stain-free properties, our data suggest that CLIN-m triple antibiotic nanofibers might be a viable alternative to minocycline-based antibiotic pastes

Synthesis and characterization of CaO-loaded electrospun matrices for bone tissue engineering

Objectives To synthesize and characterize biodegradable polymer-based matrices loaded with CaO-nanoparticles for osteomyelitis treatment and bone tissue engineering. Materials and methods Poly(ε-caprolactone) (PCL) and PCL/gelatin (1:1, w/w) solutions containing CaO nanoparticles were electrospun into fibrous matrices. Scanning (SEM) and transmission (TEM) electron microscopy, Fourier Transformed Infrared (FTIR), Energy Dispersive X-ray Spectroscopy (EDS), contact angle (CA), tensile testing, and antibacterial activity (agar diffusion assay) against Staphylococcus aureus (S. aureus) were performed. Osteoprecursor cell (MC3T3-E1) response (i.e., viability and alkaline phosphatase expression/ALP) and infiltration into the matrices were evaluated. Results CaO nanoparticles were successfully incorporated into the fibers, with the median fiber diameter decreasing after CaO incorporation. The CA decreased with the 0addition of CaO, and the presence of gelatin made the matrix very hydrophilic (CA = 0°). Increasing CaO concentrations progressively reduced the mechanical properties (p≤0.030). CaO-loaded matrices did not display consistent antibacterial activity. MC3T3-E1 cell viability demonstrated the highest levels for CaO-loaded matrices containing gelatin after 7 days in culture. An increased ALP expression was consistently seen for PCL/CaO matrices when compared to PCL and gelatin-containing counterparts. Conclusions Despite inconsistent antibacterial activity, CaO nanoparticles can be effectively loaded into PCL or PCL/gelatin fibers without negatively affecting the overall performance of the matrices. More importantly, CaO incorporation enhanced cell viability as well as differentiation capacity, as demonstrated by an increased ALP expression. Clinical significance CaO-loaded electrospun matrices show potential for applications in bone tissue engineering

Memory Controlled Sequential Self Attention for Sound Recognition

In this paper we investigate the importance of the extent of memory in sequential self attention for sound recognition. We propose to use a memory controlled sequential self attention mechanism on top of a convolutional recurrent neural network (CRNN) model for polyphonic sound event detection (SED). Experiments on the URBAN-SED dataset demonstrate the impact of the extent of memory on sound recognition performance with the self attention induced SED model. We extend the proposed idea with a multi-head self attention mechanism where each attention head processes the audio embedding with explicit attention width values. The proposed use of memory controlled sequential self attention offers a way to induce relations among frames of sound event tokens. We show that our memory controlled self attention model achieves an event based F -score of 33.92% on the URBAN-SED dataset, outperforming the F -score of 20.10% reported by the model without self attention. Index Terms: Memory controlled self attention, sound recognition, multi-head attention

Tetracycline-incorporated polymer nanofibers as a potential dental implant surface modifier

This study investigated the antimicrobial and osteogenic properties of titanium (Ti) disks superficially modified with tetracycline (TCH)-incorporated polymer nanofibers. The experiments were carried out in two phases. The first phase dealt with the synthesis and characterization (i.e., morphology, mechanical strength, drug release, antimicrobial activity, and cytocompatibility) of TCH-incorporated fibers. The second phase was dedicated to evaluating both the antimicrobial and murine-derived osteoprecursor cell (MC3T3-E1) response of Ti-modified with TCH-incorporated fibers. TCH was successfully incorporated into the submicron-sized and cytocompatible fibers. All TCH-incorporated mats presented significant antimicrobial activity against periodontal pathogens. The antimicrobial potential of the TCH-incorporated fibers-modified Ti was influenced by both the TCH concentration and bacteria tested. At days 5 and 7, a significant increase in MC3T3-E1 cell number was observed for TCH-incorporated nanofibers-modified Ti disks when compared to that of TCH-free nanofibers-modified Ti-disks and bare Ti. A significant increase in alkaline phosphatase (ALP) levels on the Ti disks modified with TCH-incorporated nanofiber on days 7 and 14 was seen, suggesting that the proposed surface promotes early osteogenic differentiation. Collectively, the data suggest that TCH-incorporated nanofibers could function as an antimicrobial surface modifier and osteogenic inducer for Ti dental implants

Online model-based redesign of experiments for improving parameter precision in continuous flow reactors

Online model-based redesign of experiments (OMBRE) techniques reduce the experimental effort substantially for achieving high model reliability along with the precise estimation of model parameters. In dynamic systems, OMBRE techniques allow redesigning an experiment while it is still running and information gathered from samples collected at multiple time points is used to update the experimental conditions before the completion of the experiment. For processes evolving through a sequence of steady state experiments, significant time delays may exist when collecting new information from each single run, because measurements can be available only after steady state conditions are reached. In this work an online model-based optimal redesign technique is employed in continuous flow reactors for improving the accuracy of estimation of kinetic parameters with great benefit in terms of time and analytical resources during the model identification task. The proposed approach is applied to a simulated case study and compared with the conventional sequential model-based design of experiments (MBDoE) techniques as well as the offline optimal redesign of experiments

A Study on the Transferability of Adversarial Attacks in Sound Event Classification

An adversarial attack is an algorithm that perturbs the input of a machine learning model in an intelligent way in order to change the output of the model. An important property of adversarial attacks is transferability. According to this property, it is possible to generate adversarial perturbations on one model and apply it the input to fool the output of a different model. Our work focuses on studying the transferability of adversarial attacks in sound event classification. We are able to demonstrate differences in transferability properties from those observed in computer vision. We show that dataset normalization techniques such as z-score normalization does not affect the transferability of adversarial attacks and we show that techniques such as knowledge distillation do not increase the transferability of attacks